In-situ animal behavior classification using knowledge distillation and fixed-point quantization

We explore the use of knowledge distillation (KD) for learning compact and accurate models that enable classification animal behavior from accelerometry data on wearable devices. To this end, we take a deep complex convolutional neural network, known as residual network (ResNet), teacher model. ResNet is specifically designed multivariate time-series classification. to distill datasets into soft labels, which consist predicted pseudo-probabilities every class each datapoint. then labels train our significantly less student models, are based gated recurrent unit (GRU) multilayer perceptron (MLP). The evaluation results using two real-world show accuracy GRU-MLP improves appreciably through KD, approaching further reduce computational memory requirements performing inference trained via utilize dynamic fixed-point quantization (DQ) an appropriate modification graph considered models. implement both unquantized quantized versions developed KD-based embedded systems purpose-built collar ear tag devices classify in situ real time. Our evaluations corroborate effectiveness KD DQ improving efficiency in-situ